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Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH (S) FROM 
THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1 .136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If the period for reply specified above is less than thirty (30) days, a reply within the statutory minimum of thirty (30) days will be considered timely. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S.C. § 133). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1 .704(b). 

Status 

!)□ Responsive to communication(s) filed on . 

2a)D This action is FINAL. 2b)K This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) ^ Claim(s) 1-30 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1-4. 9. 1 1-14. 19.21-24 and 29 is/are rejected. 

7) E3 Claim(s) 5-8. 10. 15-18.20.25-28 and 30 is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) K The specification is objected to by the Examiner. 

10) [3 The drawing(s) filed on 30 March 2001 is/are: a)M accepted or b)Q objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

11) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 
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1 .□ Certified copies of the priority documents have been received. 
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3. Q Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
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DETAILED ACTION 

Specification 

1 . The disclosure is objected to because of the following informalities: 

In paragraph 1 ? the space holders for the application No. and filing date should be 
replaced with "09/822,549" and "March 30, 2001", respectively. 
Appropriate correction is required. 



Claim Rejections - 35 USC §103 

2. The following is a quotation of 35 U.S. C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

3, Claims 1, 2, 11, 12, 22, and 22 are rejected under 35 U.S.C 103(a) as being 
unpatentable over Suito et al», US 6,014,167 in view of Nakano et aL, US 6,094,220. 

In regard to claims 1 and 1 1, Suito discloses a system and method comprising: 

a storage device having stored therein one or more routines for determining the 
suitability of a video image for use by an object tracking system (see column 5, lines 48- 
50: It is implied the tracking signal processing circuit contains a storage device to store 
the program to perform the processing as illustrated in figure 5); and 

a processor (see figure 2, element 1 1) coupled to the storage device that when 
executing the one or more routines: 
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preprocesses a video image from a video camera to separate one or more objects 
to be tracked from the rest of the video image and compute statistics for the one or more 
objects to be tracked and the rest of the video image (see column 10, lines 26-40); 

and generates color statistics for the saturation and hue (see column 12, lines 55- 
61 and column 14, lines 10-16). 

The Suito reference does not disclose that the processor: 

generates a quality measure based on the statistics for the one or more objects to 
be tracked and the rest of the video image that indicates the suitability of the video image 
for use by an object tracking system; and 

tunes said video camera to increase said quality measure beyond a threshold. 

Nakano et aL, US 6,094,220, discloses an image pickup apparatus that generates a 
quality measure based on the statistics for the one or more objects to be tracked and the 
rest of the video image that indicates the suitability of the video image for use by an 
object tracking system (see column 4, lines 17-21 : The quality measure of an object 
extracting condition is determined based on the hue and saturation of the object) and 
tunes said video camera to increase said quality measure beyond a threshold (see column 
4, lines 25-30: The microcomputer controls the chromaticity of the object, when it 
changes to fall below a predetermined value, to restore it to the initial condition of the 
white balance control). 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to have been motivated to modify Suito et aL, US 6,014,167 in view of Nakano 
et aL, US 6,094,220 to have the processor generate a quality measure based on the 
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statistics for the one or more objects to be tracked and the rest of the video image that 
indicates the suitability of the video image for use by an object tracking system and tunes 
said video camera to increase said quality measure beyond a threshold in order to perform 
white balance control with higher accuracy when the distribution of chromaticity of the 
object changes. 

In regard to claims 2 and 12, Suito et al., US 6,014,167 in view of Nakano et al., 
US 6,094,220, discloses the system and method of claims 1 and 11, respectively. The 
Suito reference discloses wherein preprocessing comprises: 

computing color statistics for said one or more objects to be tracked (see 
column 12, lines 55-61 and column 14, lines 10-16); 

removing said one or more objects to be tracked from a background of the 
video image (see figure 1 1, lines 35-40); and 

computing color statistics for said background of the video image (see 
column 10, lines 47-50). 

In regard to claim 21, the Suito reference discloses the system and method for 
performing the operations discloses limitations disclosed in claim 12, as explained above 
in regard to claims 1 and 11. It is implied the tracking signal processing circuit of the 
Suito reference contains a machine-readable medium to store the program to perform the 
processing as illustrated in figure 5 (see column 5, lines 48-50). The Suito reference does 
not disclose that the processor: 
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generates a quality measure based on the statistics for the one or more objects to 
be tracked and the rest of the video image that indicates the suitability of the video image 
for use by an object tracking system; and 

tunes said video camera to increase said quality measure beyond a threshold. 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to have been motivated to modify Suito et al., US 6,014,167 in view of Nakano 
et al., US 6,094,220 to have the processor generate a quality measure based on the 
statistics for the one or more objects to be tracked and the rest of the video image that 
indicates the suitability of the video image for use by an object tracking system and tunes 
said video camera to increase said quality measure beyond a threshold in order to perform 
white balance control with higher accuracy when the distribution of chromaticity of the 
object changes. 

In regard to claim 22, Suito et al., US 6,014,167 in view of Nakano et al., US 
6,094,220, discloses the machine-readable medium of claim 21 . The Suito reference 
discloses wherein preprocessing comprises: 

computing color statistics for said one or more objects to be tracked(see column 
12, lines 55-61 and column 14, lines 10-16); 

removing said one or more objects to be tracked from a background of the video 
image (see figure 1 1, lines 35-40); and 

computing color statistics for said background of the video image (see column 10, 
lines 47-50). 
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4, Claims 3, 4, 13 and 14 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Suito et al., US 6,014,167 in view of Nakano et al., US 6,094,220, as applied to claiml2 
above, and further in view of Funayama et al., US 6,389,155. 

In regard to claims 3 and 13, Suito et al., US 6,014,167 in view of Nakano et al., 
US 6,094,220, discloses the system of claims 2 and 12, respectively, wherein said 
computing color statistics for said one or more objects to be tracked comprises: 

identifying the objects to be tracked (see Suito: column 10, lines 45-60). 

The Suito and Nakano references do not disclose calculating mean and variance 
values for the hue and saturation of the one or more objects to be tracked. 

Funayama et al., US 6,389,155, discloses and image processor that calculates 
mean and variance values for the hue and saturation of the one or more objects (see 
column 1 1, lines 57-59) to use for calculating the probability density functions used to 
identify an object in a region of an image (see column 1 1, line 66 to column 12, line 26). 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to have been motivated to modify Suito et al., US 6,014,167, in view of Nakano 
et al, US 6,094,220, and further in view of Funayama et al., US 6,389,155, to have the 
processor calculate mean and variance values for the hue and saturation of the one or 
more objects to calculate the probability density functions in order to identify an object in 
the region of an image where the object to be tracked is located. 

In regard to claims 4 and 14, Suito et al, US 6,014,167 in view of Nakano et al., 
US 6,094,220, and further in view of Funayama et al, US 6,389,15514, discloses the 
system of claims 3 and 13, respectively. The Suito reference discloses wherein said 
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identifying comprises aligning the at least one or more objects to be tracked with one or 
more rectangles projected onto the video image (see figure 7B, object frame). 

In regard to claims 9 and 19, Suito et al., US 6,014,167 in view of Nakano et al., 
US 6,094,220, discloses the system of claims 2 and 12, respectively, wherein said 
computing color statistics for said background of the video image comprises: 

calculating a mean value for hue of the background; and calculating a mean value 
for saturation of the background. 

The Suito and Nakano references do not disclose calculating a mean value for hue 
of the background; and calculating a mean value for saturation of the background. 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to have been motivated to modify Suito et al., US 6,014,167, in view of Nakano 
et al, US 6,094,220, and further in view of Funayama et al, US 6,389,155, to have the 
processor calculate mean values for the hue and saturation of the background to calculate 
the probability density functions in order make the search region of the object smaller by 
eliminating areas of the background from the region of interest. 

In regard to claim 23, the Suito and Nakano references disclose the system and 
method for performing the operations discloses limitations disclosed in claim 21, as 
explained above in regard to claims 1 and 11. It is implied the tracking signal processing 
circuit of the Suito reference contains a machine-readable medium to store the program to 
perform the processing as illustrated in figure 5 (see column 5, lines 48-50). The Suito 
and Nakano references do not disclose calculating a mean value for hue of the 
background; and calculating a mean value for saturation of the background. 
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It would have been obvious to one of ordinary skill in the art at the time of 
invention to have been motivated to modify Suito et al., US 6,014,167, in view of Nakano. 
et al., US 6,094,220, and further in view of Funayama et al., US 6,389,155, to have the 
processor calculate mean values for the hue and saturation of the background to calculate 
the probability density functions in order make the search region of the object smaller by 
eliminating areas of the background from the region of interest. 

In regard to claim 24, Suito et al., US 6,014,167 in view of Nakano et al., US 
6,094,220, discloses the machine-readable medium of claim 23. The Suito reference 
discloses wherein said identifying comprises aligning the at least one or more objects to 
be tracked with one or more rectangles projected onto the video image (see figure 7B, 
object frame). 

In regard to claim 29, Suito et al., US 6,014,167 in view of Nakano et al., US 
6,094,220, discloses the machine-readable medium of claim 22, respectively, wherein 
said computing color statistics for said background of the video image comprises: 

calculating a mean value for hue of the background; and calculating a mean value 
for saturation of the background. 

The Suito and Nakano references do not disclose calculating a mean value for hue 
of the background; and calculating a mean value for saturation of the background. 

It would have been obvious to one of ordinary skill in the art at the time of 
invention to have been motivated to modify Suito et al., US 6,014,167, in view of Nakano 
et al, US 6,094,220, and further in view of Funayama et al., US 6,389,155, to have the 
processor calculate mean values for the hue and saturation of the background to calculate 
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the probability density functions in order make the search region of the object smaller by 
eliminating areas of the background from the region of interest. 

Allowable Subject Matter 
5. Claims 5-8, 10, 15-18, 20, 25-28, and 30 are objected to as being dependent upon a 
rejected base claim, but would be allowable if rewritten in independent form including all of the 
limitations of the base claim and any intervening claims. 

In regard to claims 5-8, 15-18, and 25-28, the prior art does not disclose 
comprising all the limitations of the claim, specifically: 

identifying the selected pixel as a pixel belonging to the selected object if the 
difference between hue of the pixel and mean hue of the selected object is within an 
allowable range for hue, the difference between saturation of the pixel and mean 
saturation of the selected object is within an allowable range for saturation, and the 
horizontal and vertical distances of the pixel from the center of the selected object are 
within an allowable range for distance; and 

identifying a pixel as belonging to the background of the video image if the 
difference between hue of the pixel and the mean hue of the selected object is not within 
said allowable range for hue, the difference between saturation of the pixel and the mean 
saturation of the object selected is not within said allowable range for saturation, and the 
horizontal and vertical distances of the pixel from the center of the selected object are not 
within said allowable range for distance. 
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In regard to claims 10, 20, and 30, the prior art does not disclose comprising all 
the limitations of the claim, specifically: 

maximizing the saturation of each of the one or more objects to be tracked; 
minimizing the saturation of the background of the video image; 

maximizing the hue difference between the one or more objects to be tracked; and 

maximizing the average hue difference between the one or more objects to be 
tracked and the background of the video image. 

Conclusion 

6. The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. The following art discloses object tracking systems: 

US 6,394,557, 

US 6,075,557, 

US 6,760,465, 

US 2004/0201719. 



Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Gevell Selby whose telephone number is 703-305-8623. The 
examiner can normally be reached on 8:00 A.M. - 5:30 PM (every other Friday off). 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Andrew Christensen can be reached on 703-308-9644. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 
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Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 
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